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INTRODUCTION 

The  functions  $Lx\    and  )Ux)  are  said  to  be  orthogonal  to 
one  another  on  the  closed  interval  £a,b}  if 

b 


Jo.  <K*>vV0O  h  ~-  c1 


Or  if  x  assumes  only  discrete  values  this  condition  can  be 
written 


5[cjj(x)  too 


-  o 


i    •  •  •  , 


where  S  denotes  summation  over  the  given  values  Xq,  Xp 

xn-l  °^  x» 

It  was  Fourier  who  first  used  an  expansion  in  a  series 

of  orthogonal  functions  in  his  treatment  of  trigonometric  series. 
The  first  expansion  in  orthogonal  polynomials  was  preformed  by 
Legendre.   In  Legendre's  polynomials  the  variable  x  is  a  contin- 
uous variable  on  a  closed  interval  1-1,1],   Orthogonal  polyno- 
mials with  respect  to  a  discontinuous  variable  xq,  x^,  ...  , 
xn-l  were  deduced  by  Tchebysheff  who  treated  the  particular 

case  of  two  orthogonal  polynomials  with  respect  to  equidistant 

2 

variables.   R.  A,  Fisher  (1921)  was  one  of  the  first  to  use 

a  numerical  method  of  curve  fitting  by  means  of  orthogonal 


2Angus  E.  Taylor,  Advanced  Calculus,  p.  722 
Charles  Jordan,  "Approximation  and  Graduation  According 
to  the  Principle  of  Least  Squares,"  Annals  of  Mathematical 
Statistics.  1932,  3:257 


polynomials. 

This  report  will  illustrate  how  a  system  of  orthogonal 
polynomials  with  respect  to  a  discrete  variable  can  be  obtained 
and  how  some  methods  for  obtaining  them  are  derived.   It  will 
also  show  how  these  polynomials  can  be  very  useful  in  curve 
fitting. 


NUMERICAL  METHOD  FOR  GENERATING  ORTHOGONAL  POLYNOMIALS 

This  method  utilizes  ordinary  finite  differences  (Stanton, 
196l).   It  must  therefore  be  assumed  that  xQ,  x-,,  ...  ,  x  -, 
differ  by  some  constant  h.   They  may  then  be  coded  so  that 
the  values  are  taken  to  be  0,  1,  ...  ,  n-1. 

Using  the  method  of  differencing,  an  equation  for  P  ,  a 
polynomial  of  degree  r,  can  be  written 

(1) 

?r   -    A"+  XA'  +  J^iJl.  Aa  t  .  .  .  ty(«-0...U-rt,l  ^ 

where  x=0,  1,  ...  ,  n-1  and  A  represents  the  k'th  ordinary 
difference  of  Pp  with  respect  to  an  initial  value,  Pr(0). 

A  statement  that  Pr  is  orthogonal  to  powers  of  x  up  to 
(r-1)  can  be  written 

(2) 

jl  (x+.iUxta.) . . .  (K+<j_-n  ip  J  -  o, 

where  q=l,  2,  ...  ,  r  and  the  sum  is  over  the  values  of  x 
from  0  to  n-1.   Now  substituting  the  expression  for  P  from 
equation  (1)  this  condition  is 

luiKu*K..(>u1-.)]LnxA'K..  v*'»-'S,(*~rH>A"K 

If  these  two  terms  in  the  summation  of  equation  (3)  are  com- 
bined and  the  resulting  expression  is  summed  a  general  term 
would  be 


C  r  f**l)(  X4Q.1  .  .  .  t\\-c^-i\ \(  X-|\  .  ,  .  U-pV  1  )  a  ■>  ~j 


To  sum  this  term  multiply  numerator  and  denominator  by  the 
quantity  (x+q)-(x-p).   This  then  changes  expression  (if)  to 


J<LL(x  +  iKX-p)]p!llUv^-0..  .  (Xt-i)X(X-i)..,(ViiV)\]uUj;)-U-p)]  ), 


(5) 


and  multiplying  by  the  quantity  (x+q)-(x-p)  in  the  numerator 
and  subtracting  the  quantity  (x-p)  from  (x+q)  in  the  denomin- 
ator the  expression  is 


£ 


(«fp)  pi  U^^lUV^-^  ,,.UtOxU-i)...(x-pfl)-(X^-ih..(Ki-lU(t-ii...(*-pl 


!6) 


The  term 


(q+pjpj 


does  not  contain  the  index  x  .   It  may  there- 


fore be  factored  out  of  the  summation  leaving 


(7), 


l^+P*  pi  MLixt5[';(xvii\...(x+i\xh-i)...(x-pi.i\-(xv^-i)...cxH)x(x-n...a-p 

Proforming  the  indicated  summation, 

oJq^n^2t^rH--pJ^XlrPjtl)__^      (q-D(q-2)...(-p+l)(-p) 
q+l)Tq7Tq=l  h~rv- 


Tn^+-qi^-3^-)^^lnipJ^^        n^^~(n^q-)Tr^-n-l^pJIn-2-p; 

(n-l+q)  (n-2+q) . . .  (n-p+1)  (n-p)  ^1^2+qUxi^^^ 
the  remaining  term  is  -(q-1) (q-2). . . (-p+1) (-p)  +  (n-l+q) (n-2+q). . . 
(n-p+1)  (n-p)  multiplied  by  ..  A     ,   However,  (q-1)  (q-2). . . 

*  SI  Sr  *  Sr  J 

(-p+l)(-p)  mu3t  contain  a  factor  of  zero.   This  can  be  seen 
by  referring  to  (7)  and  observing  that  (x+q-1)  is  decreased 


by  one  until  the  quantity  (x-p)  is  obtained.   The  factor  x 
must  therefore  be  contained  in  this  expression,  and  with  the 
substitution  of  x=0  the  quantity  must  also  equal  zero. 

The  general  term  expressed  by  (lj.)  can  now  be  written 

or  as  a  further  simplification, 

(h^-.U  A?  (9) 


oi  (*>-  p-01  (<5_tp) 

Using  this  derived  general  term  the  condition  for  orthogonal- 
ity expressed  by  (2)  or  (3)  can  be  replaced  by  the  sum  of  the 
general  term  as  p  goes  from  0  to  r.   Written  algebraically, 
expression  (2)  or  (3)  is  equivalent  to 

7  AP  (10) 

where  q=0,  1,  ...  ,  r. 

It  can  be  seen  from  equation  (10)  that  the  condition  of 
orthogonality  is  only  dependent  on  A  ,  all  other  terms  on  the 
right  side  of  equation  (10)  being  positive.   If  a  term  can 
therefore  be  found  for  A?  which  satisfies  equation  (10),  it 
can  be  substituted  into  equation  (1)  and  the  resulting  expres- 
sion can  be  used  to  generate  orthogonal  polynomials  for  suc- 
cessive values  of  r. 

The  quantity  A   may  be  found  in  the  following  manner. 
A  polynomial  function  U  of  degree  r  may  be  written 


Ui  -  C  U.ViWx+aO  .  .  .  Ui-r) .       (11) 

Substituting  a  given  value  for  x  the  function  is 

(12) 

Using  Lagrange's  interpolation  formula  ( Stanton, 1961) , 

With  equation  (10)  in  mind  x=-q  is  now  substituted  in  equation 
(13)  v/hich  gives 


(1U 


Now  if 


.f-p  »p 


u f>  '         K-P-i)i         ' 


(15) 


(16) 


(ll|.)  becomes  c 

IL^'-H)'  5_t^ti)...('^r)  1  pi i(n-  p-iM  (pfq_)  * 

Referring  back  to  the  original  equation  of  Ux  (11),  it  is 
apparent  that  U_q  is  equal  to  zero  for  values  of  q=l,  2,  ...  , 
r.   The  resulting  equation  is 

\-       /-  &"  (17) 

C  ~  (- 1 )  qj_(  <j_  t  n  .  -  .  U  +  r  )  Z  p  H  h  -  p  -  n  I  ( p  i-  ^  ) 

P-o 

Di -riding  both  sides  of  equation  (17)  by  the  non  zero  quantity 


r 
(-1)  q(q+l). . .  (q+r),  the  equation  becomes  the  condition  of 

orthogonality  first  derived  in  (10),  ie. , 

ry         aP (18) 

p^o 
This  condition  for  orthogonality  is  satisfied  when,  from 
equation  (if?), 

p-r  .  (19) 

A        ?    U-P)i      '    * 

Up  is  given  by  (12),  and  in  order  to  hold  to  the  convention 
of  the  coefficient  of  x  being  unity  in  a  system  of  orthogonal 
polynomials,  C  is  set  equal  to 

(Pi)"  (20) 


( a  r )  I  ( v>  -.  r-  i- )  I 

p 

Rewriting  the  expression  for  A  with  Up  and  C  replaced  by  the 

stated  quantities, 

?-c  (21) 

f  _  (v^-p-)V.(pir)i  .  (pi)  L-\) 
A  p!  (r-p)l    (vw-i)l  (ar)l  ' 

p 

Substituting  this  quantity  for    A     in  (1)    and  evaluating 
the  terms  for  successive  values  of  p=0,    1,    ...    ,   r,    it  can  be 

seen  that 


p  PUn-r-iil  «Pl|    ^      II  (r-i)|  (h-r-Dl  MDI       *    r 

•  •  •  » 

(-'  f"   (/win  (-ii-r)l  (n  ^ 

■II  (f-.;  :  ■;  (v-,-r-  i  j|  Ur)|  al        a(^"1)     +"     ,     .    .     . 


(22) 


8 


Or  expressing  Pr  in  summation  notation  the  general  recursion 
formula  is 

The  assumption  made  at  the  beginning  of  this  section 
is  also  assumed  fox*  the  recursion  formula,  namely  the  x's 
differ  by  some  constant  h. 


ALGEBRAIC  METHOD  FOR  GENERATING  ORTHOGONAL  POLYNOMIALS 

In  the  method  presented  here  no  assumptions  need  be  made 
about  the  variable  x.  However,  if  the  assumption  of  constant 
differences  between  the  x's  is  made  the  arithmetic  procedures 
are  greatly  reduced. 

The  expression  used  for  P  is 

(1) 

r=  1  A**5  ~-  ArXr  i  A1Mr' i....  +  A.  . 

s=  * 
where  r=0,  1,  ...  ,  n-1.   The  condition  of  orthogonality  con- 
sidered is 

sIxk(Pr)l=0,  (2) 

where  k=0,  1,  ...  ,  r-1,  and  the  summation  is  over  all  values 
of  x.  Substituting  (1)  into  (2)  the  condition  of  orthogonal- 
ity becomes 

xK'(ArXrvA,.,xr"  f-. .  +A.)I,=  Q  . 

Now  letting  k=0,  1,  ...  ,  r-1,  the  set  of  r  equations  obtained 
is 

k=0      S(ArxI,+Ar_1xr"1+...+A0)=0,  (lj.) 

k=l      S(Arxr+1+Ar_1xr+...+A0x)=0, 

k=2      S(Arxr+2+Ar-1xr+1+...+A0x2)=0, 

•  • 

k=r-l    S(Arx2r"1+A^1x2l>-2+.:.+A0xr-1)-0. 
Sunning  the  left  side  of  the  r  equations  and  dividing  both 
sices  by  n  the  set  of  equations  can  be  expressed  as 


10 

l\vMr   +Ar-i,uP_,  -V  .  -  „+Ao-  C  t 

•                                                • 

Ar,Ua'M+Af.,,l(aiua+  -  -  .  *A.lLi-  O, 

where  Mk   is  "the  k'th  moment  about  the  origin  of  x.   Consid- 

ering the  original  equation  of  Pr  (1)  and  the  r  equations 

given  by  (5),  a  set  of  r+1  equations  may  be  written  which 

have  r+1  unknowns,  Ar,  Ar-1,  ...  ,  AQ.   If  the  assumption  is 

made  that  Ar=l,  this  set  of  homogeneous  equations  is 

U'.-B.VAr  *    xp-  A.-,  +  .-.'.>>      A*  --  o,       (6) 

MrAr    +'  ilr-i  Ar-j  1  -  •  .-  +"    Ao  -  O, 

XUAr  ••-    Alp  Ar-,  -\      .      •  .4;.;>li,A."=   O, 

C                                                                                 ft 

For  this  set  of  equations  to  have  a  non-trivial  solution,  the 

determinant  of  the  coefficient  matrix  must  equal  zero.  Writ- 

ten algebraically,  this  statement  is 

U'-fi.)     XP-'   f     ...   1 

(7) 

/A,"*          /Up-I       y'Up-J,     ...        1 

Mr*\       AAr    ,Ui--i        ,  Mi 

=  O 

V 

*               *                                * 

M*M       Ma^--^  Ma<-3  •   .   g  /Apm 

This  statement  of  equality  can  be  used  to  obtain  the  n  orthog- 

onal polynomials  ?v   which  satisfy  equation  (2).   Replacing  the 

11 


given  values  for  r  in  equation  (7)  the  following  results  are 
obtained: 

(8) 


r=0, 


r=l, 


r=2, 


K°  ~  ?0     ^  C 


—7 


p 


X'-fi 

X° 

-  o 

,«> 

.Aa<, 

f-  fc 

X* 

x° 

/Wo. 

X(, 

Mo 

Mi 

>u* 

A, 

--  \ 


='     Pi  -  x-,u, 


-  o 


o 


'i         yv 


r=3, 


where 


<VP. 

x* 

X' 

xa 

/LU 

XU 

Xl, 

/Uo 

/U  H- 

il  3 

XLW* 

/U. 

A* 

XU 

jUa 

>U* 

-  o 


i  -  Mi  *MiM++ki-M*M+-AMMiM 
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The  derived  expressions  for  Pr,  r=0,  1,  2,  3,  and  the 
generating  equation  (7)  are  true  for  any  spacing  of  the  var- 
iable x.   If  the  spacing  is  constant  the  equations  (7)  and  (8) 
can  be  simplified  by  subtracting  the  mean  "x  from  each  variable 
xl>    x2>    •••  *    xa*   This  v/ould  make  ,uK   the  k'th  moment  about 
the  mean,  x.   The  odd  moments  JUz*n    ,  i=0,  1,  ...  ,  are  then 
zero.   Substituting  A^,  =0  and  Xu  -1  in  the  expressions  for 
Pq,  Pj_,  P 2»  and  Pi,  the  resulting  equations  are 

K  -  1 , 

R  -  x . 

The  x's  in  these  equations  (9)  are  the  original  x»s  minus  "x. 
Pjj_  can  be  obtained  by  equation  (7),  but  if  the  XU-i  +  i  are  set 
equal  to  zero,  the  equation  is  simplified  to 

(10) 


-     O 


K1  P4 

x5 

K^ 

'X' 

i 

ki\ 

o 

/Ux 

G 

i 

G 

xu 

o 

XU 

G 

XU 

0 

jU4 

o 

,iu 

0 

XU 

0 

/u 

o 

lluX^utiUJii-Xl  XU  ~ii\) 
To  obtain  a  generating  function  for  the  case  of  equal 
spacing  of  x  the  observation  is  made  that 
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Po  - 

x°. 

(11) 

p,  -- 

x', 

r,  - 

)C  v  e,,o 

x°. 

I3  = 

X3  f  C  3 . ; 

xi 

v 

P«  -- 

*+  K*» 

e 

f  C^o  X  . 

Considering  the  equations  express* 

3d  by  (11)  and  the 

determin- 

ant  in  equation  (7) 

with  ^.3-t) 

=o, 

,  a  general 

express 

iion 

can 

be  written 

for  P  . 

When  i  is 

odd 

(12) 

'  aj    \    T  v^^/j 

i,-*x'-H'(U. 

,*>-4   X   4-  . 

.•  ^  T  V>i, 

X 

and  when  i 

is  even 

(13) 

U  r     X 

"  1  0  ■  • 

-.1  K°      V  Cl,  t 

--^  a   r.  . 

.  +  <U 

oX° 

• 

Again  oons: 

Ldering  the  equations  expressed  by 

(11)  it 

can 

be 

seen  that 

i?, 

--  x*- 

p  _  p 

Po, 

(14) 

XP* 

-  X3  t-C^oX-  Pi 

^£ 

,  i  X  t  C.^/  0  X 

-  P»  +  < 

X}  0  " 

> 

xP, 

=  X4tC, 

^vC^-- 

ft 

*■&,■- c^ 

}Mc, 

a  L  4;  -2 

or 

-  c,., 

■V"* 

u) 

1 0 

xP4- 

=  Lh  t  D<.,  1  1  <o- 

1  t 

V  irh  i  0-3 

*  .  . 

*  > 

(15) 

where  from 

equation 

(14) 

14 

If  Do  p  were  expanded  it  would  be  found  to  equal  zero. 

'  1+3 

Equation  (15)  expresses  x(Pi)  as  the  sum  of  "2  orthog- 

i+2 
onal  polynomials  if  i  is  odd,  and  — p~  orthogonal  polynomials 

if  i  is  even.  To  obtain  a  generating  function,  (15)  is  sub- 
stituted in  the  expression 

(16) 

5u  Pi  PK)='S(  B.+,  &  +  &,&&  t'O^ft-.  3  V-..I; 

If  i+l^-k  or  k+l^-i  this  expression  will  sum  to  zero  because 
of  the  orthogonality  condition  on  the  P's.   However,  if  suc- 
cessive values  of  k=i-3,  i-5,  i-7,  ...  were  substituted  in 
the  left  side  of  equation  (16),  and  the  P's  were  commuted 
and  summed,  a  term  would  be  left  of  the  form: 

S(X  Po  Pi-,  )  ^(PiV.R.-a  t  b.J.t   +  DiAUiit«  )= 

bi.,  s(?A) 


Prom  the  fact  that  the  sum  of  xPiP^  is  equal  to  zero  for  one 

p 

arrangement  to  ?±   and  Pk  while  equal  to  some  quantity  D,  CTS(P„) 

j-i  g   q 

when  the  P's  are  commuted,  it  can  be  inferred  that  the  D,  ,  !s 
j>l  must  be  zero.   If  i=k  the  quantity  on  the  left  in  expres- 


1$ 


2 
sion  (16)  would  be  S(xP.,),  and  examining  the  right  side  of  the 

equation  it  can  be  seen  that  this  would  sum  to  zero.   The  only 

poasible  combination  of  i  and  k  left  to  consider  is  k=i+l  and 

k=i-l  for  which  the  expression  would  be 

S(xP„p„J-$(Rt,  +  &MM  +  ...)=  S(£,), 


and 


or 


Hence 


Ui-'  "      5(P.-,  )         "        S(R.a-,)  SlKTi 


with  all  other  D*  .=0. 

Substituting  these  values  of  D*  ,  in  equation  (15),  the 
equation  becomes 

x  ?■    -  ?,     t    ^  P-"  )     P 

which  implies 


u   -  1 


P.     -   v  P   -  S  (  ?<*   ^       ? 


(21) 
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Equation  (21)  is  a  recursion  formula  for  Pi+1  in  terms  of 
lower  order  orthogonal  polynomials. 

This  recursion  formula  can  be  used  quite  easily  as  it 
stands.   However,  if  an  expression  is  needed  for  P^+t  in 
terms  of  n,  StP^)  must  be  evaluated  for  the  general  case. 
This  can  be  done  with  the  aid  of  the  material  from  Section 
One. 

From  Section  One,  equation  (10),  the  condition  for  orthog- 
onality of  P  to  a  polynomial  of  degree  r-1  is 

(22) 
P 

,   q=  0,  1,  . ..  ,  r. 


If  in  equation  (22)  q  is  set  equat  to  r+1,  the  results  would 

2 

be  the  expression  for  the  sum  of  Pr.   This  can  be  seen  by 

referring  back  to  equation  (2),  Section  One.  Algebraicly  the 
expression  would  be 

(23) 

JUr  ^   £.   (vvip-Ot  (pt.-'hl  • 

Also,  it  can  be  seen  from  expressions  (llj.),  (15),  and  (16)  of 
Section  One  that 

(2k) 

U-^o  ~-  C(-r)C-rMi)  .  .  .  (I), 


or 


U-(p„>=  Ct-if  t\  , 


and 
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{ 


or 


■tp  |qpH  )  1     %  .  AP 


[/         -  /  ,f   (arfri  )  1     4  ,  A( 

lA_oni}~u)  pi  L     P!  (vn-  p-i  )l  (ptrf  i)         ' 

where  -x=q=r+l.      Setting  the  two  equations  of   (2lj.)    equal  the 


results  are 

C  r>  _  (arf  i)l  {j — n aL 


(25) 


PI  L     PI  LM-p-|  J  I  ^p 

pro     I 


And  solving  for 


I  <*> 


4 y, 

/  p  1  (  a  -  p  - 1 )  [    ( f>  i-  r  t-  » 

t>3o 


it  is  found  equal  to 

i*  /I  ,    ■  x4  (27) 

Or  h  )  I    (^r)i  (arti)i  (n-r-i)l 

p 

To  obtain  the  needod  expression  for  S(P  )  the  expression  (26) 

and  (27)  are  multiplied  by  (n+r).   The  resulting  expression  is 

(28) 


(  r  l  )f  (^  tr)j 


(.ID!  Ur+.i;)!     (n-r-i)l 
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...V-r1)  . 

Hence,    from   (19), 

(29) 

and  the  recursion  formula  v/ould  be 

(30) 

P.     -    v  P  _  l**-i»):-    p 

• 
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ORTHOGONAL  POLYNOMIALS  FOR  UNEQUAL  INTERVALS 

One  method  was  shown  for  generating  orthogonal  polyno- 
mials for  any  interval  in  Section  Two.   Equation  (7)  of  that 
section  was  derived  with  no  restrictions  on  the  independent 
variable  x.   It  can  be  seen  from  that  equation  that  if  n  is 
large,  even  of  size  if,  the  work  involved  becomes  prohibitive. 
The  methods  presented  in  this  section  will  not  completely 
solve  this  problem.   In  fact  there  is  no  method  available  at 
present  v/hich  treats  unequal  intervals  in  which  the  work  is 
not  prohibitive  for  large  n. 

Robson  (1959)  presented  a  method  in  which  he  stated  Pr 
can  be  constructed  recursively  from  the  relation 


4-o  fcs  I 


(1) 


where  c  is  the  normalizing  constant 

c;  --  i  Uf-  I  Pv  i  *■  ft  ) . 

The  polynomials  generated  by  this  method  will  be  orthonormal 
satisfying  the  conditions: 

Grandage  (1958)  in  an  answer  to  a  question  by  G.  W. 
Snedecor  illustrates  another  method.   The  method  described  by 
Grandage  is  similiar  to  the  use  of  equation  (7),  Section  Two. 
It  requires  the  solution  of  r  linear  equations  for  the  construe- 
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tion  of  Pr.   The  method  utilized  is  to  construct  the  succeed- 
ing polynomials  subject  to  the  constraints  of  orthogonality. 
For  example,  if  P1  is  set  equal  to  x+a  the  given  values  of  x 
could  be  substituted  into  the  equation  for  P1#  The  condition 

S(Pq?1)=0  could  than  be  utilized  to  solve  for  a,  where  Pq=1. 

2 

Similarlly  in  the  equation  ?2=x  +b2X+a2»  &2   anc*  ^2  can  ^e 

solved  for  subject  to  the  constraints  S(PqP2)=0  and  SfP-^J^O. 

Wishart  and  Metakides  (1953)  illustrates  a  method  for 
orthogonal  polynomial  fitting  by  abbreviated  schemes  for  invert- 
ing matrices.   The  method  described  in  this  artical  can  also 
be  used  if  there  are  unequal  weights  at  the  different  levels. 
The  method  is  clearly  explained  in  the  artical  cited  and  will 
not  be  covered  in  this  report. 

In  the  Appendix  an  example  is  shown  for  the  derivation 
of  orthogonal  polynomials  for  some  artificial  data  by  the 
three  methods  presented  in  this  section. 
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UNDERLYING  ASSUMPTIONS  IN  CURVE  PITTING  BY  ORTHOGONAL 

POLYNOMIALS 

Given  n  points  (x,y)  where  x  is  the  independent  variable 
and  y  is  the  dependent  variable  it  may  be  desirable  to  fit  a 
polynomial  of  degree  r  to  these  observations,  where  r  is  less 
than  n. 

The  general  regression  polynomial  that  is  to  be  fit  Is 
of  a  general  form 

(1) 


-    Z.   r\  l>    V'k)   f 


<-o 


where  P.  is  a  polynomial  of  degree  i  subject  to  the  conditions 
of  orthogonality,  and  A.^  is  determined  by  the  method  of  least 
squares. 

The  P^'s  can  be  obtained  by  any  of  the  methods  presented 
in  this  report  with,  of  course,  the  restriction  on  the  inter- 
val of  the  x's  determining  which  particular  method  is  used. 
These  methods  are  simply  a  way  to  determine  the  P^'s  so  that 
the  following  equations  are  satisfied 

(2) 

S  ( R  P,  V=  o , 
SCR;  K)-  O, 
S(P,    P,)--o, 
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Or  in  general  S(PiPj)=0  where  i=0,  1,  ...  ,  J-l,  and  S  is  the 
summation  over  all  values  of  x.   Pq  is  set  equal  to  1. 

To  evaluate  the  coefficients  A^,    the  method  of  least 
squares  is  utilized,  that  is,  the  deviations  of  y  about  the 
regression  polynomial  are  to  be  minimized  by  the  selection  of 
the  A^'s.   The  sum  of  the  deviations  of  y  about  the  regression 
polynomial  is  expressed  by  the  equation 

(3) 

sii  l,  -  luX)  -  su  l,  -  (.a.?,  i- a. p.  f . . .  f  Ar?r  in . 

The  A^s  are  now  selected  to  minimise  equation  (3).  Taking 
repeated  derivitives  with  respect  to  AQ,  A-j_,  ...  ,  Ar,  the 
equations  obtained  are 

■f^---  SUlr(A0Ra,p,  +  ...+A,.i:?)(-Po)]l, ik) 


j  A,- 

Multiplying  through  the  right  side  of  (!}.)  by  the  respective 
P-^'s,  the  expressions  are  greatly  simplified  because  of  the 
orthogonality  of  the  P's.  Setting  the  derivitives  equal  to 
zero,  the  equations  are 


SIijR-yLfN  b, 


(5) 
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SLR  -A.P,a]-  c 


S[^?„  -  ArP/Ko. 


Summing  the  given  expressions  in  (f?)  and  solving  for  the  A's 

(6) 


A     =    3 (up,  ) 


A     -    SfuP.I 

^      ii?f)     • 

In  testing  the  goodness  of  fit  of  the  derived  polynomials, 
the  deviations  from  regression  sum  of  squares  is  needed.   This 
is  obtained  from  equation  (3). 

If  the  expression 

c      r.        \  (7) 

u  -  I    Ai  P.  i     ' 


c-o 

is  squared,  the  quantity  would  be 

if  t  At  P/HA'P*  t ...  +  AJ  P*  -  ^CvjA.P-  ^A,P,  f<8). . 

TllArPp]   tl-LU.RP,    +A.AaP.Pa.+...+An.iR,Prl. 


2k 


However,  when  summed,  the  cross  product  terms  involving  P^ 
P*  would  go  to  zero  because  of  the  orthogonality  of  Pj_  and 


P  .,  leaving  the  term 


^u^ik\?:-x{riMh?,]}. 


(9) 


Utilizing  the  expression  for  Aj.  shown  in  (6),  the  quantity 
becomes 

-  Xu  l\o\c    -  .iu  A  i  I  i  ~  ...  "  3j  M  An  I  r  i# 

Summing  over  all  values  of  x  and  factoring  the  constant  values 
-out  of  the  summation,  the  results  are 


$(\£)    \    Ao%^: 


f 


A,  4^ 


-AArStuPrV; 

Combining  terms,  the  sum  of  squares  of  the  deviations  from 

regression  can  now  be  expressed 


It  can  be  seen  from  equation  (12)  that  the  reduction  in 
the  residual  sum  of  squares  is  caused  by  the  term  AjPj,,  by 
the  quantity  ZA^StyP^,  or  by  an  equivalent  expression,  ZSUfpf). 
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CONCLUSION 

If  n  points  of  (x,y)  are  to  be  fit  by  a  polynomial  of 
degree  r  where  r  is  less  than  n,  the  method  used  would  be  to 
derive  successive  polynomials  of  degree  0,  1,  ...  ,  r,  stop- 
ping where  contributions  made  by  the  succeeding  polynomials 
are  deemed  insignificant*   It  is  realized  that  if  r  is  not 

postulated  in  advance  a  slight  bias  is  introduced  in  the 

p 

estimate  of  a   (Anderson  and  Bancroft,  19^2).   If  a  method 

such  as  least  squares  is  used  to  determine  these  successive 
polynomials  all  coefficients  would  have  to  be  determined  for 
each  polynomial.   For  example,  if  linear  regression  was  first 
tried,  the  least  squares  equation  would  be 

(1) 

y=A+Bx, 

where  the  coefficients  A  and  B  are  determined  by  the  equations 

(2) 


1  y  ~-  hA   -V  B  i.  I 


\- 1 


and 


In  the  case  of  orthogonal  polynomials,  by  utilizing  the 
partition  theorem  for  the  X"2"  -distribution  (Cochran,  193£) 
the  sum  of  the  deviations  about  the  regression  polynomial  can 
be  shown  to  be  equal  to  the  sum  of  Q^,  Q2,  ...  $   0^,  variables 
which  are  stochastically  independent  and  distributed  as  %\    . 
A  valid  'F'  test  can  then  be  made  on  the  reduction  due  to  lin- 
ear regression,  then  on  the  additional  reduction  due  to  para- 
bolic regression  and  30  on  up  to  the  additional  reduction  due 
to  the  regression  of  the  polynomial  of  degree  r-1. 
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If  the  deviations  from  the  linear  case  were  found  to  be  sig- 
nificant, a  quadratic  may  be  fit  to  the  data.   The  least 
squares  equation  would  be 

(3) 

y=C+Dx+Ex  , 

where  the  letters  C,  D  and  E  are  used  to  emphasize  the  fact 
that  C  does  not  equal  A,  and  D  does  not  equal  B.   C,  D  and  E 
can  be  determined  by  the  equations 

\  u.  --  hC+  d'Iu  t  e'i  xt,         w 

ZXMi  "  C  iu  +  D  i  X-    f  hi    Xi  , 

is)        1  .C-i  **l  v-l 

IxiUi-s.Clx*  \  D'i  x!  +  El  Xo 

*.S:j  I  i-s|  t.c|  *.-Z[  * 

From  these  equations  it  can  be  seen  that  if  r  was  larger  than 
two,  the  work  involved  solving  successive  regression  polynomi- 
als could  be  prohibitive. 

The  solution  of  this  problem  of  recalculation  all  coef- 
ficients for  each  polynomial  can  be  solved  by  the  use  of 
orthogonal  polynomials.   As  stated  in  this  report,  the  regres- 
sion polynomial  can  be  written 

y--  i  a. a, 

■ 
where  ?±   is  a  polynomial  of  degree  i  subject  to  the  conditions 
of  orthogonality  and  A±   is  determined  by  the  method  of  least 
squares.   The  P»s  can  be  obtained  by  the  methods  outlined  in 
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this  report.   If  the  x's  are  evenly  spaced,  tables  have  been 
constructed  for  n  up  to  lolj.  and  ?d   (Anderson  and  Houseman, 
191+2 ).   The  coefficients,  A^,,  which  caused  the  trouble  in  other 
methods  can  be  determined  separately  in  the  case  of  orthog- 
onal polynomials  can  greatly  reduce  the  work  involved  in 
curve  fitting. 
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APPENDIX 

Given  the  set  of  ordered  pairs  (0,9),  (1,5),  (2,6), 
Ob  11),  (5,15),  (6,17),  (8,8)  and  (10,-1),  a  polynomial  is 
to  be  fit  to  this  data.   Plotting  the  points  it  appears  that 
the  desired  regression  polynomial  is  a  cubic,  Pig.  (1).   The 
form  of  the  regression  polynomial  is  then 

(1) 


y-  \  /up.. 


The  first  method  used  to  obtain  the  regression  polyno- 
will  be  that  of  Section  Three.   The  equations  needed  from 
that  section  are 

D  ,  (2) 


o 


R    --  X-Xc,, 

p    -    \-x  j_  x  LJjl*  -Xc,  k±)  r  Lkz-M>  Xu) 

p    -    tf    -jLiMtUs  tWtis+JJLsU^-MiJlJls'UXIlrMM) 


l 


ilMiuZiUiiLlu'+kt  ->Mr  -titlW-^klu) 


J 


wnere 


d-  il.?  I  XCiU  4-Xly-i/.vXU  -  -i  Ut  XU  XL. 

To  obtain  the  orthogonal  polynomials  the  values  of  the  respec- 
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tive  moments  are  substituted  into  the  given  equations.   Prom 
the  given  points  the  following  values  are  obtained 

(3) 


.U, 


> 


a,  --^--  -tt£k-. 

XI,  --  JJf  -   Iff  7-H    . 

Substituting  into  the  given  formulas  (1)  the  equations  obtained 
for  the  orthogonal  polynomials  are 

P.  - 1  .  (W 

P3  -  )'*  1  X(  *    a/sH  tHp  "—  '  *  ) 

1  <+s  A   I  TTX   : ^  ,  ,i ■ 


tfa      8 


--  X'1-  W"f):X  f 

I 


For  ease  in  further  computation  ?±   is  multiplied  by  \±,    chang- 
ing the  orthogonal  polynomials  to  the  form 
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P, 

'  -  Po  - 

i  , 

(5) 

?: 

1  -  IF?  - 

a,  x.  - 1 , 

V 

-  SP,  -- 

Sxa-?S  x  +  i 

p,' 

»./flf»' 

r  if  I  x3-a,;i1  X** 

n,iu,x--^ 

• 

In  solving  for  the  coefficients 

of  the  regression  poly- 

nomial 

the 

following 
Table  1 

table  is  set  up, 

» 

x  y 

p'o 

pi   p5 

p'i  yPo 

yPi 

yP?   yP'^ 

0  9 

1  5 

2  6 

4  11 

5  15 

6  17 
8  8 

10«-1 

1 
i 
1 
1 
1 
1 
1 
1 

-9   105 

-7    35 

-5   -19 

-1   -79 

1   -85 

3   -75 

7    -7 

11   125 

-4,308 

1,666 

4,228 

2,500 

-98 

-2,72k 

-4,676 

3,412 

21*1 

8  336  1+7,376  91,192,904  70 
1    2     8     1I4.I 

-46 

-2,594  -66,172 

2yPi 

• 

From  Section  i-'ive  the 

• 

equation  for  the  coe: 

fficients  in  the 

regression  ; 

polynomial 

i3 

1 

A. 

_  S(uP„  ) 

• 

(6) 

The  cooffic; 

ients  for  1 

;his  example  will  therefore  be 

(7) 

A 

> 

A 

z   ^3,  US 

, 

A 


k '  --  -  ^- ,  A'  --  -  \W* 


/  i.  ik  S    '  /  v 


:   the  derived  quantities,  and   P*    in  equation 

i    •    the   d   si  •<  ,  regression  polynomial  is 

y--  4f  («]  tf^fjUi-i)  f-^J(Sx-  7jx+i6i-)(8) 

t  -/.  -/iuffi,,] ( '*n  *3  -  -i  m  xa n,  ju, » -  feat ) 

If  an    1   Ls  of  variance  is  now  to  be  preformed  on 
-      the  needed  quantities  can  be  obtained  from 
(1).   ,  a  seen  in  Section  Five  the  residual  sum  of 
-  as  can  be  expressed 

SSI  l  -  5  /LP-li=  S(SY-ri  AiS(u?J 

)     ><.-o  J        i-o  I       • 

It  can  be  seen  from  tl  uation  that  the  reduction  in  the 

sidual  sum  of  squares  is  caused  by   each  successive  term, 
<%).   These  tei    AjSCyP^,  i->0  represent  the  reduction 
■  3     of   j  area  caused  by  linear  above  the  mean,  quad- 
bove  linear  and  so  on  to  the  r'th  degree  above  the 

For  the  illustrated  example  Table  (2)  shows 
respective  sum  of  squares. 
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Table  2. 


Sourse  of  variation 

J2 


Degrees  of 
freedom 


Sum  of  squares 


Total,  Zy  7 

Correction  1 

for  mean 
Deviations  from  6 

mean 

Linear  1 

regression 
Deviations  from  5 

linear  regression 

Second  degree  1 

term 
Deviations  from  l\. 

quadratic  regression 

Third  degree  1 

term 
Deviations  from  third      3 

degree  regression 


A'0s(yPb)  =  (35A)  (70)=6l2.5 
81|2-6l2.5=229.5 

A/1S(yP,1)  =  (-23/l68)(-l+6)=6.298 
229.5-6.298=223.202 

A'2s(yP2)=(-l,  297/23, 688)  (2,594)= 

ll|2.030 
223. 202-ll|2. 030=81. 172 

X,S(yP;)=(-l6,5l4-3/22, 798)  (-66,172)= 

J    J  48.016 
81.172-48.016=33.156 


The  next  method  used  will  be  Robson's,  Section  Pour. 
'If  equation  (1)  of  Section  Four  is  expanded  it  can  be  seen 

that 

Co  lo    "      I  r 

c.  P,  =  X-T, 

ca  ?,  --  xa'-p.  nx?o~  p,  i^P,, 

C3f3  ="xVrP.a8>.-Pl.ix5Pi  "  P<  H3P^. 


(9) 


The  equations  for  cQP0  and  c^  can  be  written  directly  as 


(10) 


Co  'u 


c  ?■  =  X  -  -i- 
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0^2  can  be  obtained  from  the  table, 
2 


CQP0 


Cl? 


0 
1 
2 

I 

8 
10 


0 

1 

ifc 

n 

100 


1 
1 
1 
1 
1 
1 
1 
1 


-9/2 

-7/2 

-5/2 

-1/2 

1/2 

3/2 

7/2 

11/2 


x  C0P0 


*  clpl 


CjL=:    SCciPi)  V8  V336A  2il.6  1,638/2 

By  substituting  the  given  quantities  into  equation  (8), 


c* a         /rlrrJ 


_kAl 


FF  U 


/k-33 


.^ 


T" 


--x--4x  v-i# 


Setting  up  a  similiar  table,    C3P3   can  be  obtained. 


x     cnP 


DjLQ. 


dP 


ILL 


o?P? 


X    CqPq  X    CtP 


111. 


(11) 


x  c5P 


2L2 


0 

1 

8 

6k 

12? 

216 

512 

1,000 


1 
1 
1 
1 
1 
1 
1 
1 


-9/2 

-7/2 

-5/2 

-1/2 

1/2 

3/2 

7/2 

11/2 


105/8 
35/8 
-19/8 
-79/8 
-85/8 
-75/8 
-7/8 
125/8 


VS        V5WI?        V47,376/6i|      1926     15,21+6/2      89A18/8  # 


Substituting  in  the  equation  (8) 


t^-.t-f^] 


(X 

.In  J 

1.2 

• 

1 

9- 

/  b~  a  4-  u 
^ 


(12) 
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(X^-^fx   t 


loS 


) 


r   81,41* 


"  X    /  4 1  X   '   /4 1  x    Ki   * 


If  the  c^P^'s  are  multiplied  by  \^x s  the  polynomials  ^c-jP^ 
are  identical  to  those  found  by  the  first  method  shown. 
With  Robson's  orthonormal  polynomials 


A.  --  S>(uPi). 


(13) 


If  however,  this  formula  was  changed  so  that  it  contained 
cipI  i-fc  would  be  identical  to  equation  (J?)  and  the  regression 
polynomial  formed  would  be  equivalent  to  equation  (7). 

The  last  method  to  be  used  is  that  of  Grandage.   Util- 
izing the  constraints  on  the  orthogonal  polynomials  that  the 
products  must  sum  to  zero  the  tables  are  constructed  so  that 
on  substituting  the  given  values  of  x  and  summing  the  products 
equal  zero. 


PO 


Plax+ai 


Ml. 


Pl=2P!=2x-9 


0 
1 
2 

* 

8 
10 


1 
1 
1 
1 
1 
1 
1 
1 


1+a-L 
2+&1 

?+al 
o+ai 

8+a£ 
■12+SL. 


-9 

-7 

-5 

-l 

l 

3 

7 

11 


36+8a1=0 
ai=-9/2 


* 
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It  can  be  seen  from  the  table  that  a^  is  solved  for  subject 

to  the  constraint  of  orthogonality.   P-j=2x-9, 

is  therefore 

orthogonal  to 

Pq.   In  a  similiar  manner  P2  is 

1  obtained. 

x   P0   Pi 

Pp=x  +bpx+ap       PqP? 

PnPp 

01-9 
11-7 
2   1   -5 
k   1   -1 

5  11 

6  1   3 

8   17 
10   1   11 

a2 
l+lb2+a2 

k+2b2+a2 

l6+kb2+a2 

25"J"|t>2+a2 

36+6b2*a2 

6ij.+8b2+a2 

100+10bp+ap 

2i|.6+36b2+8a2=0 

l,638+l68b2=0 

a2=36(39)-4(2^6)A(8) 

b2=- 1,638/168 

Therefore 

and 

£  =  2 P.-  8xM**  + 

1 0  >r. 

P  can  be  found  subject  to  the  constraints 

s(p0p3)=o, 

8(P1P3)»0, 

S(P2P3)=0, 

• 

from  a  similiar  table.   It  is  equal  to, 

The  orthc 

)gonal  polynomials  are  the  same 

as  those  found 
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in  the  first  method  and  therefore  the  work  outlined  for 
Obtaining  fcho  regression  polynomial  will  be  identical. 

Figure  (1)  page  I4.O  shows  the  original  set  of  points  and 
.ved  regression  polynomials.   The  graph  indicates  how 
much  the  succeeding  contributions  of  above  the  mean,  above 
linear  and  above  quadratic  make  to  the  regression  polynomial. 
If  the  x's  in  the  preceding  example  had  been  evenly 
c  >d  the  quantities  Pt_,  P2,  and  P3  evaluated  at  the  given 
x's  could  have  been  obtained  from  Anderson  and  Houseman  (I9I4.2). 

..  values  of  S(]     re  al30  listed  in  this  reference.   If 
t  i   orthogonal  polynomials  are  needed,  P-^  through  P^  are 

en  in  this  reference  and  any  higher  degree  can  be  obtained 
from  the  r     Lon  formula,  equation  (21)  or  (30)  Section 
Throe  of  this  report.   The  v/ork  needed  to  do  the  analysis  of 

.  an  e  or  that  of  obtaining  ch&   regression  polynomial  would 
hav   t  ".-..  a  fraction  of  the  time  needed  for  this  example. 

oly  .paced  x's  Table  (1)  could  have  been  set  up 
directly,   -'no  .     .ould  be  obtained  as  shown  for  the  case 
of  the  unequal  intervals  as  well  as  the  sum  of  squares  for 
successive  reduction:.;  due  to  each  term  A^P^.  1 
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ABSTRACT 

The  functions  !-|(k)   and  ]Kx)  are  said  to  be  orthogonal 
to  one  another  on  the  closed  interval  [&>*>]  lf 

X  l$U)lMl*  ~-  c. 

Or  if  x  assumes  only  discrete  values  this  condition  can  be 

written 

S(f(x)^U)  )=0  t 


•  •  •  » 


where  S  denotes  summation  over  the  given  values  x^,  x2, 
xn  of  x.   This  latter  condition  of  orthogonality,  orthogon- 
ality with  respect  to  discrete  variables,  is  the  one  dis- 
cussed in  this  report. 

The-  derivation  of  a  number  of  methods  for  generating 
orthogonal  polynomials  for  both  equal  spacing  of  the  variable 
x  and  for  unequal  spacing  are  shown.  Recursion  formulas  are 
derived  to  generate  orthogonal  polynomials  for  equal  spacing 

of  x. 

Given  the  regression  polynomial 


V-  i  L  P. , 

/     r.u  * 


where  P  is  a  polynomial  of  degree  i  subject  to  the  conditions 
of  orthogonality,  and  A^  is  determined  by  the  method  of  least 
squares,  it  is  shown  that  the  coefficients  can  be  independently 
obtained.   Utilizing  this,  numerical  methods  of  curve  fitting 


can  be  greatly  simplified. 

It  was  the  desire  of  the  writer  of  this  report  to  give 
the  reader  both  an  insight  into  the  derivation  of  orthogonal 
polynomials  and  also  some  knowledge  of  their  use  in  curve 
fitting.  The  majority  of  the  report  deals  with  the  derivation 
of  the  orthogonal  polynomials  with  respect  to  discrete  variables. 


